[Chapter 10] 10.5 The UNIX System Log (syslog) Facility
Author: Simson Garfinkel & Gene Spafford
Date: 1999-02-04T00:07:40Z

#

| Chapter 10 Auditing and Logging |

## 10.5 The UNIX System Log (syslog) Facility

In addition to the various logging facilities mentioned above, many versions of UNIX provide a general-purpose logging facility called syslog, originally developed at the University of California at Berkeley for the Berkeley sendmail program. Since then, syslog has been ported to several System V-based systems, and is now widely available. The uses of syslog have similarly been expanded.

syslog is a host-configurable, uniform system logging facility. The system uses a centralized system logging process that runs the program /etc/syslogd or /etc/syslog. Individual programs that need to have information logged send the information to syslog. The messages can then be logged to various files, devices, or computers, depending on the sender of the message and its severity.

Any program can generate a syslog log message. Each message consists of four parts:

- Program name

- Facility

- Priority

- Log message itself

For example, the message:

>     login: Root LOGIN REFUSED on ttya

is a log message generated by the login program. It means that somebody tried to log into an unsecure terminal as root. The messages's facility (authorization) and error level (critical error) are not shown.

The syslog facilities are summarized in Table 10.1. Not all facilities are present on all versions of UNIX.

Table 10.1: syslog Facilities
Name | Facility
--- | ---
kern | Kernel
user | Regular user processes
mail | Mail system
lpr | Line printer system
auth | Authorization system, or programs that ask for user names and passwords (`login, su, getty, ftpd,` etc.)
daemon | Other system daemons
news | News subsystem
uucp | UUCP subsystem
local0... local7 | Reserved for site-specific use
mark | A timestamp facility that sends out a message every 20 minutes

The syslog priorities are summarized in Table 10-2:

Table 10.2: syslog Priorities
Priority | Meaning
--- | ---
emerg | Emergency condition, such as an imminent system crash, usually broadcast to all users
alert | Condition that should be corrected immediately, such as a corrupted system database
crit | Critical condition, such as a hardware error
err | Ordinary error
warning | Warning
notice | Condition that is not an error, but possibly should be handled in a special way
info | Informational message
debug | Messages that are used when debugging programs
none | Do not send messages from the indicated facility to the selected file. For example, specifying *.debug;mail.none sends all messages except mail messages to the selected file.

When syslogd starts up, it reads its configuration file, usually /etc/syslog.conf, to determine what kinds of events it should log and where they should be logged. syslogd then listens for log messages from three sources, shown in Table 10.3.

Table 10.3: Log Message Sources
Source | Meaning
--- | ---
/dev/klog | Special device, used to read messages generated by the kernel
/dev/log | UNIX domain socket, used to read messages generated by processes running on the local machine
UDP port 514 | Internet domain socket, used to read messages generated over the local area network from other machines

### 10.5.1 The syslog.conf Configuration File

The /etc/syslog.conf file controls where messages are logged. A typical syslog.conf file might look like this:

>     *.err;kern.debug;auth.notice /dev/console
>     daemon,auth.notice           /var/adm/messages
>     lpr.*                        /var/adm/lpd-errs
>     auth.*                       root,nosmis
>     auth.*                       @prep.ai.mit.edu
>     *.emerg                      *
>     *.alert                      |dectalker
>     mark.*                       /dev/console

> NOTE: The format of the syslog.conf configuration file may vary from vendor to vendor. Be sure to check the documentation for your own system.

Each line of the file contains two parts:

- A selector that specifies which kind of messages to log (e.g., all error messages or all debugging messages from the kernel).

- An action field that says what should be done with the message (e.g., put it in a file or send the message to a user's terminal).

> NOTE: You must use the tab character between the selector and the action field. If you use a space, it will look the same, but syslog will not work.

Message selectors have two parts: a facility and a priority. kern.debug, for example, selects all debug messages (the priority) generated by the kernel (the facility). It also selects all priorities that are greater than debug. An asterisk in place of either the facility or the priority indicates "all." (That is, *.debug means all debug messages, while kern.* means all messages generated by the kernel.) You can also use commas to specify multiple facilities. Two or more selectors can be grouped together by using a semicolon. (Examples are shown above.)

The action field specifies one of five actions:[7]

> [7] Some versions of syslog support additional actions, such as logging to a proprietary error management system.

- Log to a file or a device. In this case, the action field consists of a filename (or device name), which must start with a forward slash (e.g., /var/adm/lpd-errs or /dev/console). [8]

  > [8] Beware: logging to /dev/console creates the possibility of a denial of service attack. If you are logging to the console, an attacker can flood your console with log messages, rendering it unusable.

- Send a message to a user. In this case, the action field consists of a username (e.g., root). You can specify multiple usernames by separating them with commas (e.g., root,nosmis). The message is written to each terminal where these users are shown to be logged in, according to the utmp file.

- Send a message to all users. In this case, the action field consists of an asterisk (e.g., *).

- Pipe the message to a program. In this case, the program is specified after the UNIX pipe symbol (|). [9]

  > [9] Some versions of syslog do not support logging to programs.

- Send the message to the syslog on another host. In this case, the action field consists of a hostname, preceded by an at sign (e.g., @prep.ai.mit.edu.).

With the following explanation, understanding the typical syslog.conf configuration file as shown earlier becomes easy.

`*.err;kern.debug;auth.notice /dev/console`

  This line causes all error messages, all kernel debug messages, and all notice messages generated by the authorization system to be printed on the system console. If your system console is a printing terminal, this process will generate a permanent hardcopy that you can file and use for later reference. (Note that kern.debug means all messages of priority debug and above.)

`daemon,auth.notice /var/adm/messages`

  This line causes all notice messages from either the system daemons or the authorization system to be appended to the file /var/adm/messages.

  Note that this is the second line that mentions auth.notice messages. As a result, auth.noticemessages will be sent to both the console and the messages file.

`lpr.* /var/adm/lpd-errs`

  This line causes all messages from the line printer system to be appended to the /var/adm/lpd-errs file.

`auth.* root,nosmis`

  This line causes all messages from the authorization system to be sent to the users root and nosmis. Note, however, that if the users are not logged in, the messages will be lost.

`auth.* @prep.ai.mit.edu`

  This line causes all authorization messages to be sent to the syslog daemon on the computer prep.ai.mit.edu. If you have a cluster of many different machines, you may wish to have them all perform their loggings on a central (and presumably secure) computer.

`*.emerg *`

  This line causes all emergency messages to be displayed on every user's terminal.

`*.alert |dectalker`

  This line causes all alert messages to be sent to a program called dectalker, which might broadcast the message over a public address system.

`mark.* /dev/console`

  This line causes the time to be printed on the system console every 20 minutes. This is useful if you have other information being printed on the console and you want a running clock on the printout.

> NOTE: By default, syslog will accept log messages from arbitrary hosts sent to the local syslog UDP port. This can result in a denial of service attack when the port is flooded with messages faster than the syslog daemon can process them. Individuals can also log fraudulent messages. You must properly screen your network against outside log messages.

### 10.5.2 Where to Log

Because the syslog facility provides many different logging options, this gives individual sites flexibility in setting up their own logging. Different kinds of messages can be handled in different ways. For example, most users won't want to be bothered with most log messages. On the other hand, `auth.crit` messages should be displayed on the system administrator's screen (in addition to being recorded in a file). This section describes a few different approaches.

#### 10.5.2.1 Logging to a printer

If you have a printer you wish to devote to system logging, you can connect it to a terminal port and specify that port name in the /etc/syslog.conf file.

For example, you might connect a special-purpose printer to the port /dev/ttya. You can then log all messages from the authorization system (such as invalid passwords) by inserting the following line in your syslog.conf file:

>     auth.*                 /dev/ttya

A printer connected in such a way should only be used for logging. We suggest using dot-matrix printers, rather than laser printers, because dot-matrix printers allow you to view the log line by line as it is written, rather than waiting until an entire page is completed.

Logging to a hardcopy device is a very good idea if you think that your system is being visited by unwelcome intruders on a regular basis. The intruders can erase log files, but after something is sent to a printer, they cannot touch the printer output without physically breaking into your establishment.[10]

> [10] Although if they have superuser access they can temporarily stop logging or change what is logged.

> NOTE: Be sure that you do not log solely to a hardcopy device. Otherwise, you will lose valuable information if the printer jams or runs out of paper, the ribbon breaks, or somebody steals the paper printout.

#### 10.5.2.2 Logging across the network

If you have several machines connected together by a TCP/IP network, you may wish to have events from all of the machines logged on one (or more) log machines. If this machine is secure, the result will be a log file that can't be altered, even if the security on the other machines is compromised.* To have all of the messages from one computer sent to another computer, you simply insert this line in the first computer's syslog.conf file:

>     *.*               @loghost

This feature can cause a lot of network traffic. Instead, you limit your log to only "important" messages. For example, this log file would simply send the hardware and security-related messages to the remote logging host, but keep some copies on the local host for debugging purposes:

>     *.err;kern.debug;auth.notice /dev/console
>     daemon,auth.notice           /var/adm/messages
>     lpr.*                        @loghost1,@loghost2
>     auth.*                       @loghost1,@loghost2
>     *.emerg                      @loghost1,@loghost2
>     *.alert                      @loghost1,@loghost2
>     mark.*                       /dev/console

Logging to another host adds to your overall system security: even if people break into one computer and erase its log files, they will still have to deal with the log messages sent across the network to the other system. If you do log to a remote host, you might wish to restrict user accounts on that machine. However, be careful: if you only log over the network to a single host, then that one host is a single point of failure. The above example logs to both loghost1 and loghost2.

Another alternative is to use a non-UNIX machine as the log host. The syslog code can be compiled on other machines with standard C and TCP/IP libraries. Thus, you can log to a DOS or Macintosh machine, and further protect your logs. After all, if syslog is the only network service running on those systems, there is no way for someone to break in from the net to alter the logs!

#### 10.5.2.3 Logging everything everywhere

Disks are cheap these days. Sites with sufficient resources and adequately trained personnel sometimes choose to log everything that might possibly be useful, and log it in many different places. For example, clients can create their own log files of syslog events, and also send all logging messages to several different logging hosts - possibly on different networks.

The advantage of logging in multiple places is that it makes an attacker's attempts at erasing any evidence of his presence much more difficult. On the other hand, multiple log files will not do you any good if they are never examined. Furthermore, if they are never pruned, they may grow so large that they will shut down your computers.

### 10.5.3 syslog Messages

The following tables[11] summarize some typical messages available on various versions of UNIX.:

> [11] A similar list is not available for System V UNIX, because syslog is part of the Berkeley UNIX offering. Companies that sell syslog with their System V offerings may or may not have modified the additional programs in their operating systems to allow them to use the syslog logging facility.

Table 10.4: Typical Critical Messages
Program | Message | Meaning
--- | --- | ---
halt | halted by <user> | <user> used the /etc/halt command to shut down the system.
login | ROOT LOGIN REFUSED ON <tty> [FROM <hostname>] | root tried to log onto a terminal that is not secure.
login | REPEATED LOGIN FAILURES ON <tty> [FROM <hostname>] <user> | Somebody tried to log in as <user> and supplied a bad password more than five times.
reboot | rebooted by <user> | <user> rebooted the system with the /etc/reboot command.
su | BAD SU <user> on <tty> | Somebody tried to su to the superuser and did not supply the correct password.
shutdown | reboot, halt, or shutdown by <user> on <tty> | <user> used the /etc/shutdown command to reboot, halt, or shut down the system.

Other critical conditions that might be present might include messages about full filesystems, device failures, or network problems.

Table 10.5: Typical Info Messages
Program | Message | Meaning
--- | --- | ---
date | date set by <user> | <user> changed the system date.
login | ROOT LOGIN <tty> [FROM <hostname>] |
| root logged in. |
su | <user> on <tty> | <user> used the su command to become the superuser.
getty | <tty> | /bin/getty was unable to open <tty>.

> NOTE: For security reasons, some information should never be logged. For example, although you should log failed password attempts, you should not log the password that was used in the failed attempt. Users frequently mistype their own passwords, and logging these mistyped passwords would help a computer cracker break into a user's account. Some system administrators believe that the account name should also not be logged on failed login attempts - especially when the account typed by the user is nonexistent. The reason is that users occasionally type their passwords when they are prompted for their usernames. If invalid accounts are logged, then it might be possible for an attacker to use those logs to infer people's passwords.

You may want to insert syslog calls into your own programs to record information of importance. Third-party software also often has a capability to send log messages into the syslog if configured correctly. For example, Xyplex terminal servers and Cisco routers both can log information to a network syslog daemon; Usenet news and POP mail servers also log information.

If you are writing shell scripts, you can also log to syslog. Usually, systems with syslog come with the logger command. To log a warning message about a user trying to execute a shell file with invalid parameters, you might include:

>     logger -t ThisProg -p user.notice "Called without required # of parameters"

> NOTE: Prior to 1995, many versions of the syslog library call did not properly check their inputs to be certain that the data would fit into the function's internal buffers. Thus, many programs could be coerced to accept input to write arbitrary data over their stacks, leading to potential compromise. Be certain that you are running software using a version of syslog that does not have this vulnerability.

#### 10.5.3.1 Beware false log entries

The UNIX syslog facility allows any user to create log entries. This capability opens up the possibility for false data to be entered into your logs. An interesting story of such logging was given to us by Alec Muffet:

> A friend of mine - a UNIX sysadmin - enrolled as a mature student at a local polytechnic in order to secure the degree which had been eluding him for the past four years.

> One of the other students on his Computer Science course was an obnoxious geek user who was shoulder surfing people and generally making a nuisance of himself, and so my friend determined to take revenge.

> The site was running an early version of Ultrix on an 11/750, but the local operations staff were somewhat paranoid about security, had removed world execute from "su" and left it group-execute to those in the wheel group, or similar; in short, only the sysadmin staff should have execute access for su.

> Hence, the operations staff were somewhat worried to see messages with the following scrolling up the console:

>     		BAD SU: geekuser ON ttyp4 AT 11:05:20
>     		BAD SU: geekuser ON ttyp4 AT 11:05:24
>     		BAD SU: geekuser ON ttyp4 AT 11:05:29
>     		BAD SU: geekuser ON ttyp4 AT 11:05:36
>     		...

> When the console eventually displayed:

>     		SU: geekuser ON ttyp4 AT 11:06:10

> all hell broke loose: the operations staff panicked at the thought of an undergrad running around the system as root and pulled the plug (!) on the machine. The system administrator came into the terminal room, grabbed the geekuser, took him away and shouted at him for half an hour, asking (a) why was he hacking, (b) how was he managing to execute su and (c) how he had guessed the root password?

> Nobody had noticed my friend in the corner of the room, quietly running a script which periodically issued the following command, redirected into /dev/console, which was world-writable:

>     		echo BAD SU: geekuser ON ttyp4 AT `date`

> The moral of course is that you shouldn't panic, and that you should treat your audit trail with suspicion.

10.4 Per-User Trails in the Filesystem |  | 10.6 Swatch: A Log File Tool